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Abstract 

Although functional analysis is a powerful tool for testing the function of challenging behavior, 
it is not always feasible or appropriate to include as a component of functional behavior 
assessment (FBA). Alternative experimental analysis methods are needed to inform 
individualized interventions in schools, particularly for students who engage in passive forms of 
problem behavior. We evaluated a concurrent operant analysis (COA) framework to identify 
reinforcers for appropriate replacement behaviors for four students referred for FBA and reported 
by teachers to engage in low levels of work completion. After completing two COAs per student 
(researcher-as-therapist and teacher-as-therapist), we used alternating treatments designs to 
compare the effects of an intervention matched to COA outcomes to intervention conditions that 
were not matched to COA outcomes on levels of work completion and task engagement. COA 
outcomes corresponded across therapists for three of four participants and intervention results 
validated COA outcomes for two of these participants. Although results of this initial 
investigation seem promising, more research on COA frameworks is needed to determine their 
utility to guide reinforcement-based interventions in schools. 

Keywords: concurrent operants, concurrent operant analysis, choice assessment, 
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An Initial Evaluation of a Concurrent Operant Analysis Framework 
to Identify Reinforcers for Work Completion 

The effectiveness of function-based interventions to reduce challenging behavior is well- 
documented (Goh & Bambara, 2012; Walker, Chung, & Bonnet, 2017), with evidence 
suggesting function-based interventions are more effective than those that are not matched to 
behavior function (e.g., Briere & Simonsen, 2011; Miller & Lee, 2013). Function-based 
interventions are designed from results of a functional behavior assessment (FBA). Of all the 
assessment strategies that can be included in an FBA, experimental analysis is the most rigorous, 
as it allows for hypotheses to be directly tested and confirmed (Hanley, 2012). 

To date, the analysis method with the most empirical support for informing effective 
function-based interventions is the functional analysis (Beavers, Iwata, & Lerman, 2013; Gage, 
Lewis, & Stichter, 2012). In a functional analysis, levels of challenging behavior are compared 
between two or more conditions in which specific antecedents and/or consequences of these 
behaviors are programmed. Conditions that consistently produce higher levels of problem 
behavior are identified as evoking or maintaining these behaviors. Although research on practical 
variations of the functional analysis for school settings has increased markedly in recent years 
(Lloyd, Weaver, & Staubitz, 2016), functional analyses are rarely used in practice (e.g., O’Neill, 
Bundock, Kladis, & Hawken, 2015; Roscoe, Phillips, Kelly, Farber, & Dube, 2015). 

Several barriers to completing functional analysis in schools have been identified. First, 
functional analysis requires evoking problem behavior to produce interpretable outcomes. This 
requirement can be seen as presenting unnecessary risk and disruption to instructional routines 
(Hanley, 2012; Roscoe et al., 2015). Second, relative to the population for whom functional 


analyses were originally designed (i.e., individuals with intellectual/developmental disabilities 
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who engage in self-injury; Iwata et al., 1982/1994), a wider range of students are being referred 
for FBAs in schools. In multi-tiered systems of support, students are referred for additional 
behavioral supports when they are not responsive to less intensive strategies (Crone, Hawken, & 
Horner, 2015). This means students with and without disabilities, and those who engage in a 
wide variety of behaviors that interfere with learning, may be referred for FBA (Erickson, Stage, 
& Nelson, 2006; Goh & Bambara, 2012; McKenna, Flower, Kim, Ciullo, & Haring, 2015). 
Applications of functional analysis to a heterogeneous student population—including a range of 
behavioral profiles—presents both practical and methodological challenges. 

Particularly in academic contexts, a common behavioral concern is an absence of 
appropriate behavior during instruction (e.g., noncompliance, low levels of task engagement). 
Even though these behavioral deficits do not present an immediate safety risk, chronic patterns 
have been shown to negatively impact student achievement, teacher and peer relationships, and 
educational placements (Kalb & Loeber, 2003). These behavioral deficits also present 
methodological challenges to conducting a functional analysis. One challenge relates to defining 
and delivering consequences contingent on the absence, rather than the presence, of a target 
behavior. Another challenge involves designing an appropriate control condition when task 
demands must be present to preserve the opportunity for passive problem behavior to occur 
(Thompson & Iwata, 2005). Thus, although functional analysis is a powerful tool, there are 
conditions in which school teams may find the procedures unacceptable or unlikely to produce 
interpretable results based on a student’s behavioral profile. For these reasons, alternative 
experimental analysis methods are needed to inform individualized interventions in schools. 

One alternative is a concurrent operant analysis (COA). A COA is an analysis in which 


two or more stimulus conditions are simultaneously available, with access to each condition 
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contingent on some response (Fisher & Mazur, 1997). Rather than the occurrence of problem 
behavior, patterns of choice allocation are used to inform relative preference between two or 
more conditions. Results of preference assessments using concurrent operant arrangements have 
been shown to predict reinforcer value (Fisher et al., 1992; Piazza, Fisher, Hagopian, Bowman, 
& Toole, 1996), suggesting COAs can be used to identify reinforcers for appropriate behaviors. 

COAs represent a promising alternative to the functional analysis for several reasons. 
First, assessment procedures do not involve evoking and reinforcing problem behavior, and 
interpretable outcomes do not hinge on whether consistent patterns of problem behavior are 
captured. Thus, practitioners may consider COA procedures more acceptable relative to those 
used for functional analysis (Lloyd, 2018). Second, the measurement of choice allocation (as 
opposed to problem behavior) represents a useful alternative for students who engage in 
behaviors that present measurement or procedural challenges to functional analysis. For 
example, Quigley et al. (2013) conducted a COA following an inconclusive functional analysis 
for a student with passive problem behavior. During the COA, the student consistently allocated 
his time to conditions that did not involve work, suggesting passive problem behavior was 
maintained by negative reinforcement. 

Third, COAs may be advantageous when there is reason to suspect problem behavior is 
maintained by escape—a commonly identified function for noncompliance and other problem 
behaviors occurring in the context of academic instruction (McKerchar & Abby, 2012). Because 
response effort has been shown to impact compliance in young children (e.g., Wilder, Fischetti, 
Myers, Leon-Enriquez, & Majdalany, 2013), escape alone can be insufficient to reinforce 
compliance or work completion as a higher effort alternative to problem behavior (Neef, Shade, 


& Miller, 1994). In the study by Quigley et al. (2013), researchers designed an intervention in 
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which the student chose between (a) working to earn access to adult attention and preferred items 
and (b) a break without access to these consequences. Even though escape from work was 
available as an option, the student shifted his responding toward the work alternative when work 
completion was reinforced with attention and preferred activities. In another example, Gardner, 
Wacker, and Boelter (2009) used COAs to demonstrate that two typically developing children 
with escape-maintained problem behavior chose to complete academic work when it was paired 
with high-quality attention, even when escape from demands was available for an alternative 
response. Thus, COAs may be an efficient strategy for identifying potential reinforcers for 
replacement behaviors that are more effortful than problem behavior. A final related advantage is 
that COAs have potential to inform intervention packages that do not require escape extinction 
(i.e., a procedure in which interventionists withhold escape when escape-maintained problem 
behavior occurs). This procedure is difficult for teachers to implement consistently, can require 
intrusive forms of prompting, and has been shown to produce unpleasant side effects (e.g., Ward, 
Parker, & Perdikaris, 2017). 

In contrast to the expansive literature on functional analysis, the literature on COAs 
includes a small collection of studies using varied procedures, conditions, and sequences of 
comparisons to address specific questions about study participants’ preferences (e.g., Berg et al., 
2007; Harding, Wacker, Berg, Lee, & Dolezal, 2009; Quigley et al., 2013). To date, no published 
studies have proposed or evaluated a standard COA framework to identify potential reinforcers 
for appropriate behavior. In an unpublished dissertation study, Casey (2001) developed a COA 
framework based on conditions included in a study by Harding et al. (1999). The framework 
included a series of pairings of up to eight conditions using concurrent operant arrangements to 


assess the relative value of attention, tangible, and escape as reinforcers for appropriate (i.e., 


CONCURRENT OPERANT ANALYSIS 8 


choice-making) behavior. Casey completed COAs and brief functional analyses for 23 children 
with high-incidence disabilities who were referred to an outpatient clinic for noncompliance and 
other problem behaviors. Results of brief functional analyses were inconclusive for 10 
participants, whereas results of COAs were conclusive for all participants. Moreover, for the 13 
children for whom functional analyses produced conclusive outcomes, results matched or 
partially matched those of the COA for nine children. This COA framework has been used 
successfully by state-funded technical-assistance teams who provide behavioral consultation and 
related professional development services in public school settings (e.g., Bassingthwaite & 
Casey, 2016; Mews, Bassingthwaite, & Ausenhus, 2018; Simcoe, Staubitz, Gregory, & Juarez, 
2018). However, no studies to date have attempted to replicate this COA framework and evaluate 
its utility to inform individualized interventions in school settings. 

The purpose of the current study was to evaluate the utility of a COA framework (Casey, 
2001) to identify reinforcers for work completion for students referred for FBA and reported to 
engage in low levels of work completion. The following research questions guided the study: 

1. Does the COA framework lead to an interpretable outcome (i.e., the identification of a 
potential reinforcer for work completion)? 

2. With minimal training, can teachers implement COA conditions with fidelity, and do 
student outcomes differ depending on whether research staff or teaching staff serve as 
therapists? 

3. Does an intervention based on COA outcomes produce higher levels of (a) work 
completion and (b) task engagement relative to intervention conditions that are not based 
on COA outcomes? 


Method 
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Participants 

After obtaining study approval from Vanderbilt University’s Institutional Review Board, 
we recruited student participants who (a) were in grades K-5, (b) received special education 
services or were identified by school personnel as at-risk for a disability (e.g., special education 
referral), (c) were referred for individualized assessment and intervention (i.e., FBA and 
behavior support plan), and (d) were nominated by their classroom teacher as a student who 
engaged in low levels of compliance and/or work completion during one or more instructional 
routines. Inclusion criteria were evaluated based on interviews with teachers, record reviews, and 
an initial classroom observation. We obtained informed consent from each student’s parent and 
teacher(s) and verbal assent from each participating student before collecting any study data. 

Four students and four teachers participated in this study. All four students scored in the 
Abnormal range on the Strengths and Difficulties Questionnaire (SDQ; Goodman, 2001), which 
was completed by their teachers. Demetrius was a 9-year-old Black boy in fourth grade who was 
diagnosed with attention deficit hyperactivity disorder (ADHD) and had average intellectual 
functioning. He attended a public special day school for students with emotional/behavioral 
disorders. Demetrius’s classroom teacher was a 25-year-old White woman who was in her first 
year of teaching; she had a Master’s degree in Special Education and was a Board Certified 
Behavior Analyst. Donny was a 9-year-old Black boy in fourth grade who attended a public 
elementary school. He had been referred by his school’s student support team for special 
education evaluation, but was not receiving services at the time of participation. His teacher was 
a 27-year-old White woman with a bachelor’s degree, elementary teaching certification, and 5 
years of classroom teaching experience. Charles and Braden were two 6-year-old boys who 


received special education services under developmental delay at a public elementary school. 


CONCURRENT OPERANT ANALYSIS 10 


Charles was Black and Braden was Black/American Indian. Their kindergarten classroom 
teacher was a 26-year-old White woman with 3.5 years teaching experience and certification in 
both K-6 general education and K-12 special education. Their special education teacher was a 
53-year-old White woman with 15 years of experience and certification in special education. 
Settings and Materials 

Demetrius’s COA sessions were conducted in a small conference room at his school. For 
Donny, Charles, and Braden, researcher-as-therapist COAs were conducted in empty auxiliary 
rooms in the school building and teacher-as-therapist COAs were conducted in their usual 
classrooms while other students were in Art or PE classes. Intervention sessions were conducted 
in the usual classroom during scheduled 1:1 instruction (Demetrius), at a desk in the hallway 
outside the usual classroom (Donny), and in a teacher planning room (Charles and Braden). 

We arranged each COA at a table with a line of masking tape dividing the space in half. 
We also placed lines of tape on the floor to clearly demarcate two contiguous choice areas. 
Materials associated with each choice condition (i.e., preferred items, neutral/low-preferred 
items, task materials; see Table 1) were placed on corresponding sides of the table. Materials 
used during intervention sessions included: 8.5 x 11 in. (21.59 x 27.94 cm) laminated token 
boards, three sets of 1 x 1 in. (2.54 x 2.54 cm) laminated tokens, and academic worksheets. 

We worked with each student’s teacher to select academic tasks to include in the COA 
and intervention comparisons. We asked teachers to identify tasks that were within each 
student’s skill repertoire but for which low levels of work completion were commonly observed. 
For Demetrius and Donny, we selected brief (140—180 words) fiction passages with a set of 10 
comprehension questions. Passages were on a 3"-grade reading level and accessed from 


ReadWorks®. To control for task difficulty across conditions, we randomly assigned passages to 
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intervention conditions and used the same set of 10 comprehension questions (e.g., Who is the 
main character?) across sessions. For Charles and Braden, we selected worksheets on matching 
pictures to letter sounds. To control for task difficulty across conditions, we randomly selected 
10 letters for each worksheet, and randomly assigned worksheets to conditions. During 
intervention sessions, we used a Time Timer™ to track work time and reward time. 
Measurement System and Response Definitions 

During each COA, we measured student choice allocation. We used Lily data collector 
(Tapp, 2010) on Dell™ electronic tablets to measure duration of choice allocation via timed 
event sampling. There were three mutually exclusive duration codes: Choice Area 1, Choice 
Area 2, and No Choice. Allocation to each choice area was defined as having both feet in one of 
the two choice areas. No Choice was coded when one or both feet were outside a choice area or 
on the dividing tape. Changes in choice allocation were coded based on a 3-s offset (e.g., ifa 
student left Choice Area 1, Choice Area 2 was coded only after the student remained in Choice 
Area 2 for at least 3 s). The dependent variable was the percentage of session time spent in each 
choice area, calculated as the number of seconds allocated to each area divided by the total 
session duration (s) and multiplied by 100. 

During intervention sessions, we measured work completion via permanent product and 
task engagement using timed event sampling. We measured work completion as the total number 
of questions answered or work sheet items completed in each 5-min session. We consulted with 
each student’s teacher to determine requirements for completed items. For Demetrius and 
Donny, answers to comprehension questions had to (a) begin with a capital letter, (b) end with 
punctuation, and (c) be related to passage content (i.e., contain at least one context-specific word 


related to the passage). For Charles and Braden, a letter had to be matched correctly to the 
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corresponding picture (e.g., a line drawn from the letter “B” to a picture of a bumblebee) to be 
counted as a complete response. We defined task engagement as being oriented toward the 
academic materials and demonstrating one or more of the following behaviors: (a) writing (or 
erasing) with pencil on the worksheet in a manner consistent with the academic activity, (b) 
reading aloud, (c) scanning the page and/or turning the page over, or (d) asking a question 
pertaining to the worksheet (adapted from Kamps, Wills, & Wehby, 2012). We coded 
engagement duration using a 5-s offset rule. We calculated the percentage of the work period the 
student was engaged as the total number of seconds engaged divided by the duration of the work 
period (s) and multiplied by 100. 
Experimental Design 

For each COA, we used a simultaneous treatments experimental design (Barlow & 
Hayes, 1979) and a predetermined sequence of comparisons (Casey, 2001; see Figure 1) to 
evaluate choice allocation among concurrently available conditions. We changed conditions 
when the student allocated 70% or more of the 5-min session (i.e., 3.5 min) to one choice area; 
we repeated the same condition if less than 70% of session time was allocated to one choice area. 

For each intervention comparison, we used an alternating treatments design (Barlow & 
Hayes, 1979) to compare effects of three potential intervention conditions with differing 
reinforcement contingencies on work completion and task engagement. With few exceptions, we 
completed one set of block randomized sessions (1.e., one session per condition, in a randomly- 
determined sequence) per day. We used visual analysis to evaluate response differentiation 
among the rapidly alternating conditions. We identified one condition as more effective than 
another if the level of work completion or task engagement was consistently higher across three 


or more series of conditions within the same phase (Wolery, Gast, & Ledford, 2018). We 
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hypothesized students would complete more work and be more engaged in conditions 
corresponding with their COA outcomes relative to the other conditions. 
Study Procedures 

Indirect assessments. We used a combination of open- and closed-ended interview and 
questionnaire materials (i.e., modified Functional Assessment Interview [FAI; O’Neill et al., 
1997]; SDQ [Goodman, 2001]; researcher-developed COA materials interview) to collect 
information from teachers on student problem behavior and potential contributing factors, 
preferred activities, and representative instructional activities. Teacher responses informed the 
stimuli (e.g., tangible items, forms of attention, instructional tasks) included in each student’s 
COA and intervention comparison (see Table 1). 

Concurrent operant analysis. We completed two COAs for each participant 1-2 weeks 
apart. Each COA was completed within a 1-hr visit or across two 30-min visits on different days. 
A research team member served as therapist during the first COA; the student’s teacher served as 
therapist for the second COA. For Charles and Braden, the general education teacher conducted 
the teacher-as-therapist COAs with one exception: the final session of Braden’s COA was 
completed by the special education teacher based on availability. Aside from therapist role, all 
procedures were identical across both COAs. Prior to the teacher-as-therapist COA, we held a 
brief meeting with the teacher to review COA procedures and role-played the different forms of 
interaction the teacher might be asked to provide depending on student choices. 

We replicated the branching framework used by Casey (2001) to select, compare, and 
sequence all COA conditions. The COA framework included a total of eight possible choice 
conditions (see Table 1), which were paired against one another in a hierarchical sequence to 


assess the relative value of attention, tangibles, and escape as potential reinforcers (see Figure 1; 
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Casey, 2001). Each participant was exposed to at least five choice conditions. During each COA 
session, we presented two concurrently available activities on opposite sides of a table. Before 
each session, a graduate research assistant (i.e., the facilitator) explained each available choice. 
When attention or prompts were programmed in a condition, a therapist (either a second graduate 
research assistant or the student’s teacher) was seated on the side of the table associated with that 
condition. The facilitator checked for understanding and asked the student if he had any 
questions before beginning the session. If a student asked a question, the facilitator answered him 
and demonstrated what would happen in each choice area via roleplay. The facilitator also told 
the student he could switch between the two activities at any time. The facilitator then prompted 
the student to stand at a location equidistant to each area that was marked with an X. The 
facilitator instructed the student to “Make a choice” and the session began. At least once during 
the 5-min session, the facilitator reminded the student he was free to change sides at any time. 
Intervention comparison. Prior to conducting COAs, we designed three intervention 
conditions to assess the validity of COA outcomes. Each intervention condition was based on 
one of three outcomes (i.e., tangible, attention, escape). Our goal was to create intervention 
conditions that only differed with respect to the type of reward earned for task completion. We 
programmed these reward contingencies in a token economy based on its potential to be 
incorporated in classroom instruction. Research assistants implemented intervention sessions. 
Across conditions, sessions consisted of a 5-min work period during which up to 10 tokens were 
delivered contingent on work completion. Following the work period, students exchanged the 
tokens they earned for access to a potential reinforcer. Each token was worth 30 s of access. 
Before beginning intervention, the interventionist introduced each student to the token 


board and each set of tokens, and described what they could be exchanged for. She paired tokens 
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with each reward by prompting the student to exchange each type of token and delivering 30-s 
access to the corresponding reward. The interventionist began each session with a contingency 
review to specify (a) what the student would be working for, (b) the work requirement to earn 
one token, and (c) when the student could exchange his tokens. She told the student he could do 
as much or as little work as he wanted, and asked if he had any questions. After reviewing the 
rules, the interventionist set a visual timer for 5 min and the session began. For Demetrius and 
Donny, each completed comprehension question resulted in a token delivery. For Charles and 
Braden, one token was delivered for every three worksheet items completed. If a question or 
item was partially completed, the interventionist provided a verbal reminder of the completion 
requirement. However, if the student chose not to complete work, she did not deliver any further 
prompts. She responded to student questions related to the assigned task, but did not respond to 
other bids for attention. When the 5-min work period ended, the interventionist counted the 
tokens aloud and indicated how much reward time was earned. She then set the visual timer to 
match the total reward time earned, and provided access to the reward. Following sessions in 
which the student did not complete any work, the interventionist indicated that the student did 
not earn any tokens this time, but could choose to work for tokens in the next session. 

Tangible condition. In the tangible condition, the student worked to earn tangible tokens, 
which had a yellow border and an image of each student’s preferred items. Worksheets were 
printed on yellow paper to facilitate discrimination among conditions. When tangible tokens 
were exchanged, the interventionist provided access to the same highly-preferred tangibles 
evaluated in each participant’s COA, but did not otherwise interact with the student. 

Attention condition. In the attention condition, the student worked to earn attention 


tokens, which had a green border and a speech bubble icon. Worksheets were printed on green 
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paper. When attention tokens were exchanged, the interventionist provided continuous attention 
without delivering prompts; preferred items were withheld. The type of social attention varied 
across students based on preference, which was informed by both teacher and student report. 

Escape condition. In the escape condition, the student worked to earn escape tokens, 
which were white with a black border and no icon. Worksheets were printed on white paper. 
When escape tokens were exchanged, the interventionist provided a break without attention or 
preferred items. During these breaks, she oriented away from the student. Although we 
conceptualized the escape condition to represent an intervention matched to an escape outcome, 
we also designed it to inform whether the provision of tokens alone increased work completion. 
This is because participants could freely access escape during all sessions. If a participant chose 
to complete work to earn tokens across conditions, including escape tokens that could be 
exchanged for breaks that were already freely available, this pattern might suggest the delivery 
of tokens alone (regardless of what they were exchanged for) reinforced work completion. 

Attention + Tangible condition. We added a condition for Donny based on results of the 
COA and information from the first series of intervention sessions. In this condition, Donny 
worked to earn attention + tangible tokens, which had a blue border and images of a speech 
bubble and iPad. Worksheets were printed on blue paper. When attention + tangible tokens were 
exchanged, the interventionist provided access to both preferred items and social attention (i.e., 
access to the iPad while the therapist provided comments on the games he played). 

Choice probe. For three of four participants, we completed choice probes intermittently 
or at the end of the intervention comparison. During a choice probe, the interventionist presented 
all three token types and asked the participant which kind of token he wanted to work for. When 


the participant made a choice by either naming or touching the token type, we conducted that 
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condition as described in the previous sections. We attempted a choice probe for Charles, but 
were not able to complete it due to the occurrence of problem behavior when the choice was 
presented; due to time constraints, we were unable to attempt additional choice probes. 
Inter-Observer Agreement 

For each participant, we collected inter-observer agreement (IOA) data on a minimum of 
33% and 30% of COA and intervention sessions, respectively. For COA sessions, we calculated 
point-by-point agreement on choice allocation by dividing the number of seconds in which both 
observers agreed on choice allocation by the total number of seconds in the session (300). Mean 
agreement was 98.2% (range, 92.8% -100%) across participants and conditions. For intervention 
sessions, we calculated point-by-point agreement on work completion and task engagement. To 
estimate agreement on work completion, two coders independently scored each item on the 
worksheet as complete or incomplete. Agreement on complete responses was calculated as the 
number of items both observers scored as complete divided by the total number of items. Mean 
agreement on work completion was 100% for Demetrius and Donny; 84.3% (range, 75 .0%-— 
100%) for Charles, and 96.8% (range, 90.0%—100%) for Braden. For Charles and Braden, 
disagreements occurred due to difficulty determining whether lines correctly connected 
corresponding letters and pictures when multiple lines were drawn on top of each other. To 
calculate agreement on task engagement, we divided the number of seconds in which both 
observers agreed on whether the student was or was not engaged by the total number of seconds 
in the work period (300). Mean agreement on task engagement was 97.9% (range, 94.6%-—100%) 
for Demetrius; 97.7% (range, 88.5%—100%) for Donny; 89.7% (range, 77.0%-—98.2%) for 
Charles; and 94.5% (range, 83.3%-—100%) for Braden. When disagreements occurred, we relied 


on the primary observer’s codes for data entry; however, observers discussed and resolved 
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discrepancies between sessions to prevent similar disagreements in subsequent sessions. 
Procedural Fidelity 

We used paper-and-pencil data sheets to collect procedural fidelity data during COAs. 
We collected data on the occurrence of the following set-up variables for each session: correct 
materials were in each choice area; facilitator explained rules for each choice area; and student 
began each session at the designated choice location. During each COA session, we collected 30- 
S partial interval data on student behaviors (i.e., choice allocation, item interaction), therapist 
behaviors (i.e., attention delivery, directed play prompts, academic prompts), and facilitator 
behaviors (i.e., rule reminders, redirections). Because some of these behaviors were performed 
on an as-needed basis, we coded the presence or absence of each behavior during the COA 
session. Then, following the session, we scored each interval as correct or as containing at least 
one procedural error based on the student, therapist, and facilitator behaviors coded as present. 
We summarized fidelity data as a percentage of correct implementation (i.e., number of intervals 
scored as correct divided by total number of intervals [10], multiplied by 100). We collected 
procedural fidelity data during all COA sessions. Across participants, mean procedural fidelity 
exceeded 99.0% for both researcher-as-therapist sessions (range, 88.9%-100%) and teacher-as- 
therapist sessions (range, 94.4%-100%). We collected IOA data on procedural fidelity for 47.8% 
of researcher-as-therapist sessions and 62.5% of teacher-as-therapist sessions. Mean IOA for 
procedural fidelity data was 99.7% (range, 96.4%-100%) and 99.1% (range, 96.4%-100%) for 
researcher- and teacher-as-therapist sessions, respectively. 

We used tablets to collect timed-event data on procedural fidelity during intervention 
sessions. At the beginning of each session, we coded the occurrence of the following 


interventionist set-up behaviors: (a) explained rules, (b) presented correct materials, and (c) set 
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visual timer for 5 min. During each session, we coded the start and end points of each work and 
reward period, occurrences of interventionist attention, and the presence of correct materials. We 
used these data to score whether the provision of attention and materials was correct for both the 
work period and reward period. Following each session, we used permanent product data (i.e., 
student worksheet and token board) to evaluate whether the correct number of tokens was 
delivered for work completion, and whether the total reward period duration was correct based 
on the total number of tokens earned. For each session, we calculated a percentage of total 
intervention components implemented correctly. We collected procedural fidelity data during a 
minimum of 67% of intervention sessions per condition and participant. Across participants, 
mean fidelity exceeded 97.0% for each intervention condition (session range, 86.7%-100%). We 
collected IOA on intervention procedural fidelity data for at least 50% of sessions per condition. 
Mean IOA on intervention procedural fidelity met or exceeded 93.0% for attention, escape, and 
tangible conditions (session range, 66.7%—100%). For Donny’s attention + tangible condition, 
we collected IOA on fidelity in only one session (11.0%), and agreement was 66.7% (one 
observer neglected to record occurrence of interventionist set-up behaviors). 
Results 

Demetrius’s COA results are presented in Figure 2. In both COAs, Demetrius 
consistently chose to allocate all time towards one alternative; no sessions were repeated in either 
assessment. Regardless of whether a research staff member or his teacher served as therapist, 
Demetrius consistently chose to spend time in assessment areas in which his preferred tangibles 
(i.e., iPad, LEGO™ toys) were available—even when he was directed on how to play with them. 
Based on these choice patterns and the framework shown in Figure 1, we identified access to 


tangibles as the most likely reinforcer for work completion. 
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Results of Demetrius’s intervention comparison are shown in Figure 3. The total number 
of worksheet items completed is graphed in the top panel, and the percentage of session time 
with task engagement is graphed in the bottom panel. During the first three series of conditions, 
we noted Demetrius was engaged in work for 100% of the 5-min work period in three sessions, 
yet completed no more than three comprehension questions due to his handwriting pace. Because 
we were reinforcing completed items, rather than time engaged, we were concerned Demetrius 
would not contact the relevant contingencies at therapeutic levels when writing complete 
sentences was required to earn a token. After discussing this with his teacher, we adjusted the 
response requirement such that single words or phrases could count as complete answers. 
Following this change, we observed consistent differentiation in level between the tangible 
condition and each of the other conditions for both work completion and task engagement. That 
is, in four out of six series, Demetrius was engaged in work for 100% of the time and completed 
between 5 and 10 items in the tangible condition, but was engaged at zero or near-zero levels and 
completed no items in the escape and attention conditions. In the final three choice probe 
sessions (marked with asterisks), Demetrius consistently chose the tangible condition and 
completed all 10 comprehension questions. Based on the differentiation in level for time engaged 
and work completed, and the consistent selection of tangible during choice probes, we 
interpreted results of Demetrius’s intervention comparison to support his COA outcome. 

Donny’s COA results are presented in Figure 4. One condition had to be repeated during 
each COA. In the teacher-as-therapist COA, we chose to repeat a condition even though the 70% 
response allocation was met due to a procedural fidelity error made in that session (Session 5). 
Across both assessments, Donny consistently allocated his time to conditions in which both 


attention and access to tangibles were available. When he had to choose between attention and 
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tangible, he allocated time to both choice areas, but ultimately chose attention with low-preferred 
items over no attention with high-preferred items regardless of who served as therapist. 
According to the framework shown in Figure 1, we interpreted Donny’s choice patterns in both 
COAs to indicate attention as the most likely reinforcer for work completion. 

Results of Donny’s intervention comparison are shown in Figure 5. Across the first five 
series of conditions, we observed an increasing trend in item completion and high levels of task 
engagement across conditions. During intermittent choice probes, Donny chose to earn attention 
tokens (Session 10) and tangible tokens (Session 14). Because Donny demonstrated a preference 
for both tangible and attention rewards in the choice probes (as he had during the COA), we 
hypothesized that tokens that could be exchanged for time with both adult attention and preferred 
items might be the most effective reinforcer. Thus, we developed a synthesized attention + 
tangible condition. To minimize the potential for multi-treatment interference, we alternated the 
attention + tangible condition with the original escape condition only (see second phase in Figure 
5). When we compared these two conditions, the increasing trend in item completion continued, 
and the percentage of time engaged remained high in both conditions. In a final series of choice 
probes, Donny chose the synthesized condition in four of six opportunities, and completed all 
work during those sessions. Because we did not identify consistent differentiation in level for the 
number of items completed or the percentage of time engaged, results of Donny’s intervention 
comparison provided no further evidence supporting his COA outcome. 

Results of Charles’s COAs are presented in Figure 6. In both COAs, Charles consistently 
chose to allocate all time towards one alternative; no sessions were repeated in either assessment. 
Although Charles’s choice patterns did differ between researcher-as-therapist and teacher-as- 


therapist COAs, he ultimately chose to spend time in assessment areas in which access to his 
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preferred tangibles (e.g., Play-doh, LEGO™ toys, bubbles) were available. Based on these choice 
patterns and the framework shown in Figure 1, we identified access to tangibles as the most 
likely reinforcer for work completion. 

Results of Charles’s intervention comparison are presented in Figure 7. Across all series, 
Charles completed more work and spent more time engaged in the tangible condition relative to 
the attention and escape conditions. His performance varied, however, across sessions in the 
tangible condition. Charles completed between one and 24 items and was engaged between 11% 
and 100% of work periods during tangible sessions. Unfortunately, Charles’s intervention 
comparison was cut short due to the end of the school year. We were unable to evaluate whether 
additional exposure would have led to more stable responding in the tangible condition, nor were 
we able to present choice probes to evaluate Charles’s preferred intervention. Because we did 
identify consistent differentiation in level that favored the tangible condition for both items 
completed and time engaged, we interpreted results of Charles’s intervention comparison to 
support his COA outcome. 

Braden’s COA results are displayed in Figure 8. Braden’s patterns of choice allocation 
were relatively consistent; only one condition in the researcher-as-therapist COA was repeated. 
Results were similar across COAs for the first four choice comparisons. He consistently chose to 
allocate all time in assessment areas in which his preferred tangibles were available. However, in 
the final comparison, Braden chose directed play with preferred tangibles over free play with 
low-preferred tangibles when the researcher served as therapist, yet made the opposite choice 
when the teacher served as therapist. Thus, based on the framework shown in Figure 1, the 
researcher-as-therapist COA suggested tangible as the most likely reinforcer for appropriate 


behavior, whereas the teacher-as-therapist COA suggested escape. 
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Results of Braden’s intervention comparison are presented in Figure 9. With the 
exception of the first escape session, Braden completed zero items, and was engaged at zero or 
near-zero levels, in the escape condition. During two of three attention sessions and three of four 
tangible sessions, Braden completed between 12 and 19 items and was engaged for 83% or more 
of the work period. We interpreted these data as demonstrating emerging differentiation by level 
between both the attention and tangible conditions and the escape condition. However, no 
consistent differentiation was observed between the tangible and attention conditions, and 
performance was variable. During choice probes, Braden chose tangible (Session 12) and escape 
(Session 13), yet only completed work to earn tokens in the tangible condition. As with Charles, 
the end of the school year prevented us from conducting more intervention sessions to determine 
whether increased exposure to contingencies would have produced more stable levels of 
responding within condition. We interpreted Braden’s intervention comparison to provide 
preliminary support for the researcher-as-therapist COA only. 

Assessment and intervention comparison outcomes are summarized by participant in 
Table 2. Outcomes of the researcher-as-therapist and teacher-as-therapist COAs corresponded 
for three of four participants. Of these three participants, higher levels of work completion and 
task engagement were observed in intervention conditions matched to the COA outcome than 
conditions not matched to the COA outcome for two participants (Demetrius, Charles); results of 
the intervention comparison were undifferentiated for the third participant (Donny). For Braden, 
COA outcomes did not correspond between therapists; results of the intervention comparison 
partially supported the researcher-as-therapist COA outcome. 

Discussion 


The goal of this study was to evaluate the utility of a COA framework (Casey, 2001) to 
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identify reinforcers for work completion. Although COAs have been used to inform behavioral 
interventions in previous research (e.g., Piazza et al., 1997; Quigley et al., 2013), procedures 
have varied markedly across studies. The current study differs from previous studies in its 
evaluation of a proposed framework for conducting COAs to identify potential reinforcers for 
appropriate behavior. Rather than comparing outcomes of this COA model to outcomes of 
functional analyses (Casey, 2001), the current study represents an initial attempt to validate 
outcomes of the COA framework by examining effects of matched interventions. We also 
repeated each COA with a teacher serving as therapist to evaluate the reliability of outcomes 
across implementers and the feasibility of involving teachers in this process. 

All applications of the COA framework produced interpretable outcomes with respect to 
identifying a potential reinforcer for appropriate behavior. That is, all participants complied with 
facilitator prompts to make choices and engaged in patterns of behavior that suggested an 
understanding of contingencies for each choice (e.g., actively contacted or participated in 
activities that were available on the side they chose, sampled both choices before allocating a 
majority of session time to one). With minimal preparation, teachers were also able to implement 
conditions involving adult interaction with fidelity. 

Results of the researcher-as-therapist and teacher-as-therapist COAs corresponded for 
three of four participants; for Braden, the one participant with different outcomes by 
implementer, choice allocation differed in the final session of the COA. The similarity in choice 
patterns across implementers was somewhat surprising given the different learning histories 
associated with each type of therapist. Interestingly, Braden’s COA outcomes differed by 
therapist such that he chose directed play with preferred items when the researcher served as 


therapist, yet chose the alternative (i.e., free play with low-preferred items) when the teacher 
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served as therapist. This may suggest a higher tolerance for demands presented by a novel 
therapist relative to his usual teacher. 

Results of the intervention comparison validated COA outcomes for two of the three 
participants whose COA outcomes corresponded across therapists (i.e., Demetrius and Charles). 
For Donny, however, levels of work completion were variable and showed increasing trends 
across conditions, with levels of task engagement remaining high across conditions. This pattern 
may indicate that for Donny, the delivery of tokens alone reinforced work completion, regardless 
of the activities for which each token could be exchanged. For Braden, researcher-as-therapist 
and teacher-as-therapist COAs pointed to different potential reinforcers. Results of the brief 
intervention comparison did favor the researcher-as-therapist COA, yet this may have been due 
to the fact that a research staff member implemented the intervention conditions. 

The only previous attempt to validate this COA framework was done by comparing COA 
outcomes to results of brief functional analyses (Casey, 2001). Results of COAs matched or 
partially matched those of functional analyses for nine of the 13 participants for whom 
interpretable functional analysis outcomes were identified. In the current study, COA outcomes 
were validated by intervention outcomes for 2 of 4 participants, with partial validation for a third 
participant. Even for participants who showed response differentiation, however, levels of work 
completion and task engagement were variable in the more effective intervention condition. We 
designed the intervention comparison to isolate one component of intervention that was directly 
informed by the COA (i1.e., reward delivered for tokens earned). Additional individualized 
intervention components (e.g., prompting, feedback, functional communication) may have 
produced more consistent levels of work completion. Taken together, our results indicate a need 


to further evaluate the conditions in which this COA framework can inform effective 
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individualized interventions. 

A final contribution worth noting is that the COA-informed intervention conditions did 
not require escape extinction as a treatment component. The interventionist told the student he 
could complete as much work as he wanted, and did not deliver any prompts to complete work 
during sessions. Thus, despite the availability of escape, all participants chose to complete work 
in one or more intervention condition. This outcome points to the potential utility of COAs in 
classrooms where teachers are unable or unwilling to use escape extinction. 

Limitations 

Results of this study should be considered in light of the following limitations. First, we 
did not collect initial baseline data on work completion or task engagement prior to the 
intervention comparison. Thus, we can only make comparisons among the three token-based 
interventions, and cannot draw conclusions regarding whether any of these conditions improved 
performance relative to business-as-usual. Second, with the exception of Demetrius, research 
staff implemented the intervention conditions, which may have biased outcomes toward the 
researcher-as-therapist COA for one participant for whom COA outcomes differed across 
therapist (i.e., Braden). Third, we primarily relied on teacher report to select task levels and types 
for the intervention comparison; we did not directly assess student performance on these tasks 
prior to beginning intervention sessions. However, we observed high levels of task completion in 
at least a subset of sessions across participants, suggesting these tasks were indeed in students’ 
repertoires. Fourth, although we took several steps to minimize differences in task difficulty 
across intervention conditions (see Settings and Materials), the instructional stimuli varied across 
sessions and conditions and may have contributed to variability in responding. Finally, data from 


Charles’s and Braden’s intervention comparisons were abbreviated due to teacher scheduling 
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preferences and the end of the school year. Continued exposure to intervention conditions might 
have produced more interpretable and replicable outcomes. 
Future Research 

With respect to applying this COA framework in academic settings, researchers might 
adapt one or more conditions to better represent aspects of students’ usual classroom contexts in 
future studies. For example, the directed play condition might be modified to a directed work 
condition in which a teacher provides 1:1 prompting and support as the student completes a task. 
Or, for students who value peer attention over adult attention, including a peer in one or more 
choice conditions may be needed, especially if it is feasible to incorporate peer attention in the 
behavior support plan. Procedural modifications to increase experimental control for the COA 
framework may also prove useful in future studies focusing on validating outcomes. For 
example, two or more consecutive sessions with a demonstrated preference might be required 
before moving on to the next comparison. Or, implementers might systematically counterbalance 
which condition is assigned to which choice area to rule out idiosyncratic response biases. 
Depending on outcomes of future validation studies, research on procedural modifications 
designed to maximize the feasibility of COAs for school personnel (e.g., simplified data 
collection procedures, shortened session duration) might also be warranted. 

With respect to intervention, additional research is needed to evaluate the effectiveness 
and maintenance of COA-informed interventions. In the current study, we rapidly alternated 
brief, single-component intervention conditions to efficiently assess the validity of COA 
outcomes. Other single case demonstration designs (e.g., multiple baseline designs across 
participants and/or settings; changing criterion designs) may be used to evaluate both efficacy 


and maintenance of one comprehensive, classroom-based intervention relative to business as 
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usual. If these interventions are effective, evaluations of social validity — with respect to both the 
intervention and the assessment process that informed it— will be needed to inform acceptability 
and feasibility of implementation. 
Conclusion 

Although further research on applications of COAs in school settings is clearly needed, 
the current study provides preliminary support for a COA framework (Casey, 2001) to inform 
individualized interventions to reinforce appropriate behavior. Given the barriers of 
incorporating functional analysis in school-based FBAs, alternative experimental analysis 
methods are needed—particularly for students whose primary behavior-support needs involve 
behavioral deficits. In these cases, the COA may represent a promising alternative for analyzing 


choice allocation to identify conditions most likely to support appropriate student behavior. 
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Table 1 


COA Procedures by Condition and Participant 


Participant 


Condition 


Demetrius Donny Charles Braden 


A: Demand without 
attention 


Materials 


Student sits alone and completes work independently 


Math worksheet Math worksheet Phonics worksheet Phonics worksheet 


B: Free play with 
attention and 
preferred items 


Materials 


Student plays with therapist and preferred items (no prompts) 


Play-doh, Laptop, play-doh, 
iPad and iPad and LEGO™ toys, LEGO™ toys, 
LEGO™ toys LEGO™ toys bubbles, puzzle, bubbles, puzzle, 


ball, toy car ball, toy car 


C: Directed play 
with preferred items 


Therapist delivers prompts on how to play with preferred items 
(i.e., “play my way”) 


Play-doh, Laptop, play-doh, 
Materials iPad and iPad and LEGO™ toys, LEGO™ toys, 
LEGO™ toys LEGO™ toys bubbles, puzzle, bubbles, puzzle, 
ball, toy car ball, toy car 
D: Free play with 
preferred items and Student plays alone with preferred items 
without attention 
Play-doh, Laptop, play-doh, 
Matauaie iPad and iPad and LEGO™ toys, LEGO™ toys, 
LEGO™ toys LEGO™ toys bubbles, puzzle, bubbles, puzzle, 


ball, toy car ball, toy car 


E: Demand with 


Therapist delivers prompts and assistance to complete academic task 


attention 
Materials Math worksheet Math worksheet Phonics worksheet Phonics worksheet 
F: Alone Student sits alone in area without any items or activities 

Materials n/a n/a n/a n/a 


G: Free play with 
attention and low- 
preferred items 


Materials 


Student plays with therapist and neutral or low-preferred items (no prompts) 


Pot holder, coaster, coffee filter 


H: Free play with 
low-preferred items 
and without attention 


Materials 


Student plays alone with neutral or low-preferred items 


Pot holder, coaster, coffee filter 
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Table 2 


Summary of Assessment and Intervention Comparison Outcomes by Participant 


Intervention Condition(s) with 


Participant COA-Researcher COA-Teacher Highest Work Completion 


Demetrius Tangible Tangible Tangible 

Donny Attention Attention Undifferentiated 
Charles Tangible Tangible Tangible 

Braden Tangible Escape Tangible & Attention 


Note. COA = concurrent operant analysis. 


A: Demand w/o 
attention 


B: Free play w/ 
attention and 


preferred items 


preferred items 


E: Demand w/ 
attention 


D: Free play w/ 
preferred items 
and w/o attention 


C: Directed play F: Alone (no 
w/ preferred items or 
items activities) 


G: Free play w/ 
attention and low- 
preferred items 


D: Free play w/ 
preferred items and 
w/o attention 


Attention Tangible 


C: Directed play w/ 


D: Free play w/ 
preferred items and 
w/o attention 


C: Directed play F: Alone (no 
w/ preferred items or 
items activities) 


E: Demand w/ 
attention 


D: Free play w/ 
preferred items 
and w/o attention 
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Figure 1. Decision framework for concurrent operant analysis (figure adapted from 


Casey, 2001). 
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Figure 2. Results of Demetrius’s concurrent operant analyses with researcher-as-therapist (top) and 
teacher-as-therapist (bottom). Esc = Escape; Tan = Tangible; Att = Attention; LP = Low-preferred. 
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Figure 3. Effects of intervention conditions on work completion (top) and task engagement (bottom) for 
Demetrius. Asterisks indicate choice probes. Att = Attention; Esc = Escape; Tan = Tangible. 
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Figure 4. Results of Donny’s concurrent operant analyses with researcher-as-therapist (top) and 
teacher-as-therapist (bottom). Att = Attention; Tan = Tangible; Esc = Escape; LP = low-preferred. 
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Figure 5. Effects of intervention conditions on work completion (top) and task engagement (bottom) for 
Donny. Asterisks indicate choice probes. Att = Attention; Tan = Tangible; Esc = Escape. 
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Figure 6. Results of Charles’s concurrent operant analyses with researcher-as-therapist (top) and 
teacher-as-therapist (bottom). Att = Attention; Tan = Tangible; Esc = Escape; LP = low-preferred. 
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Figure 7. Effects of intervention conditions on work completion (top) and task engagement (bottom) for 
Charles. Esc = Escape; Att = Attention; Tan = Tangible. 
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Figure 8. Results of Braden’s concurrent operant analyses with researcher-as-therapist (top) and 
teacher-as-therapist (bottom). Esc = Escape; Tan = Tangible; Att = Attention; LP = low-preferred. 
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Figure 9. Effects of intervention conditions on work completion (top) and task engagement (bottom) for 
Braden. Asterisks indicate choice probes. Esc = Escape; Tan = Tangible; Att = Attention. 


